Image recording apparatus and method, image reproducing apparatus and method, and recording medium on which image processing program is recorded

ABSTRACT

The present invention is intended to record image data having a wide dynamic range is a file together with image data having a narrow dynamic range. First, an image recording apparatus separately gradation-converts image data to be recorded into primary data having a narrow dynamic range and secondary data having a wide dynamic range. Then, the image recording apparatus calculates data that determines correlation between the primary data and secondary data and employs the calculated data as tertiary data. The image recording apparatus records the primary data and the tertiary data in a file. On the other hand, an image reproducing apparatus reads out primary data and tertiary data that were recorded in the above manner and reproduces secondary data having a wide dynamic range based on the primary data and the tertiary data.

CROSS REFERENCE TO RELATED APPLICATION

[0001] The disclosure of the following priority application is hereinincorporated by reference: Japanese Patent Application No. 2000-009371,filed Jan. 18, 2000.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to an image recording apparatus andmethod for recording image data as well as to a recording medium onwhich an image processing program for realizing the above imagerecording apparatus on a computer is recorded. The invention alsorelates to an image reproducing apparatus and method for reproducing afile generated by the above image recording apparatus as well as to arecording medium on which an image processing program for realizing theabove image reproducing apparatus on a computer is recorded.

[0004] 2. Description of the Related Art

[0005] In image recording apparatuses, usually, 8-bit-gradation (24 bitstotal in the case of 3-color data) image data are compressed andrecorded in the form of a general-purpose compressed image file. Capableof being expanded by a general-purpose image-browsing program, such ageneral-purpose compressed image file has an advantage that it can beprinted or displayed easily without using any dedicated software.

[0006] In recent years, there have been developed image recordingapparatuses such as a digital still camera that can record raw data(i.e., data obtained by merely digitizing an output of an imagingdevice).

[0007] Raw data are image data that are faithful to an output of animaging device and include fine gradation components(gradation signals).Therefore, raw data have an advantage that they are not easily impairedby complex data processing for designing, printing, and like purposesand their subtle gradation signals are not easily lost.

[0008] Incidentally, a general-purpose compressed image file of about8-bit gradations as mentioned above has a relatively narrow gradationreproduction (tone reproduction) range and allows a high-luminance-sideportion of an object to be reproduced with a gradation reproductionrange of 140% white at most. Such a general-purpose compressed imagefile has a problem that high-luminance-side/low-luminance-side gradationcomponents cannot be reproduced affluently with sufficient performance.It is desired that a general-purpose compressed image file have agradation reproduction range of up to 200% white, and if possible, 400%white.

[0009] A general-purpose compressed image file is expressed with 8-bitgradations. This results in a problem that discontinuity of gradationsbecomes conspicuous when a file is subjected to image processing such asa contrast conversion.

[0010] On the other hand, raw data as mentioned above have a largenumber of quantization bits and hence allow fine middle range signals tobe reproduced affluently with sufficient performance. However, raw dataare given gradation characteristics and a data format that are specificto hardware such as a digital still camera. In standard form,general-purpose external apparatuses such as a printer and a monitorcannot deal with such raw data. That is, raw data are associated with aproblem that it requires dedicated image processing and cannot beprinted or displayed easily by a general-purpose external apparatus.

[0011] Image data such as raw data having a large number of quantizationbits include fine middle range signals affluently. The degree of spatialredundancy of raw data is extremely lower Than 8-bit-gradation imagedata. This results in a problem that conventional image compressionmethods that eliminate spatial redundancy cannot compress raw data intoa file of a small size.

SUMMARY OF THE INVENTION

[0012] In view of the above, an object of the invention is to provide animage recording apparatus and method capable of efficiently recordingimage data having a wide gradation reproduction range.

[0013] Another object of the invention is to provide a recording mediumon which an image processing program for allowing a computer to functionas the above image recording apparatus is recorded.

[0014] Still another object of the invention is to provide an imagereproducing apparatus and method for reproducing image data that wererecorded by the above image recording apparatus.

[0015] A further object of the invention is to provide a recordingmedium on which an image processing program for allowing a computer tofunction as the above image reproducing apparatus is recorded.

[0016] The invention provides an image recording apparatus comprising afirst converting unit for converting image data into primary data havingan N-bit range according to a first gradation conversion (toneconversion) characteristic; a second converting unit for converting theimage data into secondary data having an M-bit range according to asecond gradation conversion characteristic that is lower in the degreeof level compression (knee compression) than the first gradationconversion characteristic or causes no level compression, where M isgreater than N; a correlation calculating unit for calculating data thatdetermines correlation between the primary data and the secondary dataand employing the calculated data as tertiary data; and a recording unitfor recording the primary data and the tertiary data in a file.

[0017] In the above configuration, first, primary data having an N-bitrange that is high in the degree of level compression and secondary datahaving an M-bit range (M>N) that is low in the degree of levelcompression are generated from the same image data. The secondary datais image data having a wider gradation reproduction range and moregradations than the primary data.

[0018] Then, the correlation calculating unit calculates data thatdetermines correlation between the two kinds of data and employs thecalculated data as tertiary data. Usually, the primary data and thesecondary data are very similar to each other in the manner of gradationvariation because they have been generated from the same image data.Therefore, as a result of the correlation calculation operation,redundant similarities between the two kinds of data can bediscriminated properly and tertiary data that reliably includesmeaningful variation components (e.g., gradation data in the secondarydata that does not exist in the primary data) can be obtained.

[0019] The recording unit records the thus-obtained primary data andtertiary data. The amount of recording data can be reduced properlybecause redundant similarities between the two kinds of data can bediscriminated in advance in contrast to a case of recording the primarydata and the secondary data separately.

[0020] It is preferable that the recording unit be a unit for recordingthe primary data by irreversibly compressing it, and that correlationcalculating unit expand the irreversibly compressed primary data,calculate data that determines correlation between expanded primary dataand the secondary data, and employ the calculated data as the tertiarydata. Because the primary data is compressed irreversibly, irreversiblyvariations occur in the primary data as it is compressed and expanded.In this case, since the reference (primary data) to be used forreproducing the secondary data deviates, the secondary data can nolonger be reproduced completely. In view of this, the correlationcalculating unit expands the primary data that has been compressed forrecording and thereby generates expanded primary data that is the sameas primary data to be obtained at the time of reproduction. Thecorrelation calculating unit generates tertiary data by using theexpanded primary data as a reference. Therefore, the reference to beused at the time of reproduction does not deviate and hence thesecondary data can be reproduced more precisely.

[0021] It is preferable that the recording unit record the primary datain an “image storage segment to be preferentially referred to”, which isin the file. In this case, by using a general-purpose image browsingprogram or the like, the primary data can be read out and printed ordisplayed easily in the same manner as in the case of handling aconventional image file. In other words, compatibility with conventionalimage files can be maintained.

[0022] It is preferable that the recording unit record the tertiary datain an application segment (i.e., one or a plurality of data segmentsable to be optionally added to an image file) which is in the file. Byusing an application segment, compatibility with conventional imagefiles can be maintained.

[0023] It is preferable that the first gradation conversioncharacteristic and the second gradation conversion characteristic havethe same characteristic curve in at least a part of the entire inputsignal range. By partially equalizing the first and second gradationconversion characteristics, the similarity between the two gradationconversion characteristics can be increased. Therefore, in the processof generating tertiary data by correlation calculating, the amount ofrecording data of the image file can be reduced more reliably.

[0024] It is preferable that the correlation calculating unit calculatedata relating to dissimilarity between the primary data and thesecondary data and employ the calculated data as the tertiary data. Inthis case, it is not necessary to perform complex correlation detectingcalculations and hence tertiary data can be generated at high speed. Theimage reproducing apparatus side is also given an advantage that thesecondary data can be reproduced at high speed by such simple processingas addition of the primary data and the tertiary data.

[0025] It is preferable that the recording unit compress the tertiarydata by nonlinearly quantizing it and record the compressed tertiarydata in the file. Usually, tertiary data includes such data ashigh-luminance-side or low-luminance-side gradation components. Ingeneral, the sensitivity of human vision is low to small leveldifferences in such gradation components. Therefore, nonlinearlyquantizing the tertiary data makes it possible to compress the tertiarydata in such a range that the compression is not visually discernible.

[0026] It is preferable that the recording unit compress the tertiarydata by increasing sampling increments of the tertiary data on an imagespace and record the compressed tertiary data in the file. Eliminatingpieces of the tertiary data to the level in which the visual sensitivityis low (decimation) makes it possible to compress the tertiary data insuch a range that the degradation is not visually discernible. Thismeasure is not limited to use in decimation. For example, the samplingincrements may be increased by reducing high spatial frequencycomponents of the tertiary data.

[0027] It is preferable that the recording unit divide the tertiary datainto map data indicating shapes of non-correlation regions (i.e.,regions where substantial dissimilarities exist between the primary dataand the secondary data) and data indicating values of thenon-correlation regions. Recording the tertiary data in this form makesit possible to eliminate redundant data in regions other than thenon-correlation regions and thereby compress the tertiary data. Theimage reproducing apparatus side can easily reproduce the secondary databy re-disposing the data indicating values based on the map data.

[0028] It is preferable that the recording unit record pieces of thetertiary data at non-coincidence positions (i.e., positions of an imagewhere the secondary data cannot be calculated directly from the primarydata) in the file. For example, the primary data and the secondary datacorrespond to each other approximately one to one at positions havingpixel values for which the first and second gradation conversioncharacteristics coincide with each other. At positions where thecorrelation between the primary data and the secondary data is verystrong, secondary data can be calculated to some extent from the primarydata without using the tertiary data. Therefore, with this measure, thedata amount of the tertiary data can be reduced efficiently by recordingonly pieces of the tertiary data at the non-coincidence positions. Animage reproducing apparatus (described later) discriminates thenon-coincidence positions according to the primary data as reproduced,and re-disposes the recorded tertiary data on the image space. It istherefore preferable that the image recording apparatus discriminatenon-coincidence positions in a manner that the image reproducingapparatus can find the same non-coincidence positions.

[0029] It is preferable that the recording unit compress the tertiarydata by run-length coding, entropy coding, and/or predictive coding andrecord the compressed tertiary data in the file. Usually, tertiary datais calculated according to the primary data and the secondary data ofthe same image data. Therefore, it is highly probable that the same dataappears consecutively in the tertiary data. The data size of suchtertiary data can be reduced by performing run-length coding. It ishighly probable that invalid data or the same data appears at a highfrequency in the tertiary data. The data size of such tertiary data canbe reduced by performing entropy coding. Since tertiary data iscalculated from image data having strong spatial correlation from thestart, it is highly probable that the tertiary data also becomes a dataarray having strong spatial correlation. The data size of such tertiarydata can be reduced by performing predictive coding. Performing a propercombination of predictive coding, run-length coding, and entropy codingmakes it possible to compress the tertiary data into data having an evensmaller data size.

[0030] It is preferable that the second converting unit change thesecond gradation conversion characteristic in accordance with a featureof the image data.

[0031] In general, image data that are produced by imaging exhibit oneof the following various features depending on the object, the imagingconditions, and the illumination conditions:

[0032] Including many high-luminance-side gradation components.

[0033] Including many low-luminance-side gradation components.

[0034] Including many intermediate-luminance gradation components.

[0035] Including gradation components in a wide luminance range from thelow-luminance side to the high-luminance side.

[0036] The above configuration makes it possible to judge features ofthe image data such as size or deviation of a input signal range, andchange the second gradation conversion characteristic in accordance withthe feature. This makes it possible to generate secondary data thatreflects the feature of the image data faithfully. To faithfullyreproduce such secondary data afterwards, it is preferable that theprimary data and the tertiary data be recorded together.

[0037] It is preferable that an image processing program for realizing,on a computer, the functions of the first converting unit, the secondconverting unit, the correlation calculating unit, and the recordingunit be generated and recorded on a recording medium.

[0038] It is preferable that the functions of the first converting unit,the second converting unit, the correlation calculating unit, and therecording unit be converted into steps and that the steps be executedsequentially as an image recording method.

[0039] On the other hand, the invention provides an image reproducingapparatus for reproducing a file generated by the above image recordingapparatus reading unit for reading primary data and tertiary data from afile generated by the above-described image recording apparatus; andsecondary data calculating unit for reproducing secondary data based onthe primary data and the tertiary data. The tertiary data that is readout in this manner includes data that determines correlation between theprimary data and the secondary data. Secondary data (not necessarily thesecondary data itself but may be data that is close to the secondarydata at the time of recording and that has a wide gradation reproductionrange and a large number of signal levels) is reproduced based on thistertiary data and the primary data.

[0040] It is preferable that first the reading unit read the primarydata, and the tertiary data on non-coincidence positions, and that thesecondary data calculating unit discriminate the non-coincidencepositions according to pixel values of the primary data. For example,the non-coincidence positions may be discriminated by judging whetherthe pixel value of the primary data belongs to a non-coincidence portionof the first and second conversion characteristics. The reading unitdisposes the tertiary data at the non-coincidence positions and performpositioning between the primary data and the tertiary data. Thesecondary data calculating unit reproduces the secondary data based onthe primary data and the tertiary data that corresponds to the primarydata in pixel positions.

[0041] It is preferable that the secondary data calculating unitlevel-compress (knee-compress) the secondary data to be data which has arange that is gradation-reproducible by an external apparatus, andoutput the level-compressed data. One use of such secondary data is dataprocessing for printing and designing. For such data processingpurposes, it is preferable that the reproduced secondary data be outputas it is in M-bit-gradation form. On the other hand, there is anotheruse in which users can enjoy a wide gradation reproduction range andaffluent gradation variations with an external apparatus (e.g., adisplay apparatus or a printing apparatus). However, such externalapparatuses do not give standard support to such M-bit-gradation. Inview of this, the image reproducing apparatus outputs the secondary dataafter level-compressing it so that the data has a luminance rangegradation-reproducible by an external apparatus. This level compressionallows users to enjoy high-quality features of the secondary data easilywith the external apparatus.

[0042] It is preferable that an image processing program for realizing,on a computer, the functions of the reading unit and the secondary datacalculating unit be generated and recorded on a recording medium.

[0043] It is preferable that the functions of the reading unit and thesecondary data calculating unit be converted into steps and that thesteps be executed sequentially as an image reproducing method.

BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The nature, principle, and utility of the invention will becomemore apparent from the following detailed description when read inconjunction with the accompanying drawings in which like units aredesignated by identical reference numbers, in which:

[0045]FIG. 1 is a block diagram showing the configuration of a digitalstill camera 11;

[0046]FIG. 2 is a flowchart showing an image recording process accordingto a first embodiment of the present invention;

[0047]FIG. 3 is a flowchart showing an image reproducing processaccording to the first embodiment;

[0048]FIG. 4 is a graph showing gradation conversion characteristicsused in the first embodiment;

[0049]FIG. 5 is a flowchart showing an image recording process accordingto a second embodiment of the invention;

[0050]FIG. 6 is a flowchart showing an image reproducing processaccording to the second embodiment;

[0051]FIG. 7 is a graph showing gradation conversion characteristicsused at the time of recording in the second embodiment; and

[0052]FIG. 8 is a graph showing gradation conversion characteristicsused at the time of reproduction in the second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0053] Preferred embodiments of the present invention will behereinafter described with reference to the accompanying drawings.

[0054] Embodiment 1

[0055] A first embodiment is directed to a digital still camera.

[0056]FIG. 1 is a block diagram showing the configuration of a digitalstill camera 11 according to this embodiment.

[0057] As shown in FIG. 1, an imaging lens 12 is mounted on the digitalstill camera 11. The imaging surface of an imaging device 13 is disposedin the image space of the imaging lens 12. Image data produced by theimaging device 13 are supplied to a buffer memory 16 via a signalprocessing unit 14 and an A/D conversion unit 15 and stored temporarilyin the buffer memory 16. An input/output terminal of the buffer memory16 is connected to a data bus 17. A CPU 18, a frame memory 19, a memorycard 20, an image output interface 21, etc. are also connected to thedata bus 17. An output terminal of the frame memory 19 is connected to amonitor 22.

[0058] 1. Image Recording Process

[0059]FIG. 2 is a flowchart showing an image recording process accordingto the first embodiment.

[0060] The image recording process according to the first embodimentwill be described below in order of step numbers shown in FIG. 2.

[0061] Step S1:

[0062] An image signal produced by the imaging device 13 is supplied tothe A/D conversion unit 15 via the signal processing unit 14.

[0063] The A/D conversion unit 15 converts the image signal into14-bit-gradation data by linearly quantizing the image signal in adynamic range of 0-400% white. The A/D conversion unit 15 stores the14-bit-gradation data in the buffer memory 16 (temporary storage).

[0064] Step S2:

[0065] The CPU 18 performs gradation conversion of γ1 (see FIG. 4) onthe 14-bit-gradation data and obtains 8-bit-gradation data. The CPU 18stores the 8-bit-gradation data in the buffer memory 16 (temporarystorage).

[0066] Step S3:

[0067] The CPU 18 judges whether the luminance difference between“background luminance in an image space peripheral portion” and “mainobject luminance at the image space center” of the 14-bit-gradation datais larger than a threshold value. If the luminance difference is greaterthan or equal to the threshold value, the CPU 18 judges thathigh-luminance-side gradation components exist and hence importanceshould be given to high-luminance-side gradation expression and advancesthe process to step S4. On the other hand, if the luminance differenceis smaller than the threshold value, the CPU 18 judges that the dynamicrange is not wide and importance should be given to low-luminance-side,subtle gradation expression and advances the process to step S5.

[0068] Step S4:

[0069] The CPU 18 performs gradation conversion of γ2 (see FIG. 4;having a characteristic in which importance is given tohigh-luminance-side gradation expression) on the 14-bit-gradation dataand obtains 12-bit-gradation data. The CPU 18 stores the12-bit-gradation data in the buffer memory 16 (temporary storage). Then,the CPU 18 advances the process to step S6.

[0070] Step S5:

[0071] The CPU 18 performs gradation conversion of γ3 (see FIG. 4;having a characteristic in which importance is given tolow-luminance-side gradation expression) on the 14-bit-gradation dataand obtains 12-bit-gradation data. The CPU 18 stores the12-bit-gradation data in the buffer memory 16 (temporary storage). Then,the CPU 18 advances the process to step S6.

[0072] Step S6:

[0073] The CPU 18 calculates differences between the 12-bit-gradationdata and the 8-bit-gradation data and employs the calculated differencesas difference data. The difference data thus calculated is stored in thebuffer memory 16 (temporary storage). (The 8-bit-gradation data, the12-bit-gradation data, and the different data correspond to primarydata, secondary data, and tertiary data, respectively.) When12-bit-gradation data is generated by using the gradation conversioncharacteristic γ3, an offset of 2 bits occurs between the12-bit-gradation data and the 8-bit-gradation data. In this case,difference data is calculated after eliminating the offset between thetwo kinds of gradation data by multiplying the 8-bit-gradation data by 4(2 bit shift left).

[0074] Step S7:

[0075] The CPU 18 determines image regions having extremely highluminance according to the 8-bit-gradation data. Image regions havingextremely high luminance need not be reproduced finely because thevisual sensitivity is low for those regions. Therefore, in the imageregions for which the visual sensitivity is low, the CPU 18 decimatesthe samples of the difference data into about {fraction (1/2)} or{fraction (1/4)}. Where irreversible compression is performed on the8-bit-gradation data, to make a result of the image region judgmentconsistent with an image region judgment result that will be obtained atthe time of reproduction, it is preferable that the CPU 18 perform theimage region judgment according to temporary, expanded 8-bit-gradationdata.

[0076] Step S8:

[0077] The CPU 18 compresses the difference data by performingpredictive coding (e.g., DPCM). Then, the CPU 18 further compresses thedifference data by performing run-length coding. Then, the CPU 18 stillfurther compresses the difference data by performing entropy coding.

[0078] Step S9:

[0079] The CPU 18 combines the data into a single file in the followingmanner and records the file in the memory card 20.

[0080] Compressed data of 8-bit-gradation data . . . An image storagesegment of a general-purpose image file (e.g., a JPEG file)

[0081] Compressed data of difference data . . . An application segmentof the general-purpose image file

[0082] γ information (i.e., information indicating the kind of thegradation conversion characteristic that was used for the conversioninto the 12-bit-gradation data).

[0083] The recording process of the digital still camera 11 completesupon execution of step S9.

[0084] 2. Image Reproducing Process

[0085]FIG. 3 is a flowchart showing an image reproducing processaccording to the first embodiment.

[0086] The image reproducing process according to the first embodimentwill be described below in order of step numbers shown in FIG. 3.

[0087] Step S11:

[0088] The CPU 18 reproduces 8-bit-gradation data from an image filethat is recorded in the memory card 20.

[0089] Step S12:

[0090] The CPU 18 judges whether difference data exists in the imagefile recorded in the memory card 20. If difference data exists in theimage file, the CPU 18 advances the process to step S13. On the otherhand, if difference data does not exist in the image file, the CPU 18judges that the image file is a conventional one and finishes thereproducing process.

[0091] Step S13:

[0092] The CPU 18 extracts the difference data and γ information fromthe image file recorded in the memory card 20. Further, the CPU 18sequentially performs, on the difference data, expanding operationscorresponding to entropy coding, run-length coding, and predictivecoding in this order.

[0093] Step S14:

[0094] According to the 8-bit-gradation data, the CPU 18 judges imageregions for which the visual sensitivity is low. In those image regions,the CPU 18 infers, according to surrounding difference data etc.,difference data that was eliminated at the time of recording.

[0095] Step S15:

[0096] The CPU 18 restores 12-bit-gradation data by adding thedifference data to the 8-bit-gradation data.

[0097] If it is judged according to the γ information that the gradationconversion characteristic γ3 was used at the time of recording, the CPU18 restores 12-bit-gradation data by adding the difference data to the8-bit-gradation data as multiplied by 4 (2 bit shift left).

[0098] Step S16:

[0099] The CPU 18 outputs the thus-generated 12-bit-gradation datatogether with the γ information etc. to the external system via theinterface 21. Further, the CPU 18 converts the 12-bit-gradation datainto 8-bit-gradation data by performing simplified gradation conversion(e.g., one having a characteristic γ4′ shown in FIG. 8) for monitordisplay or printing in an external apparatus on the 12-bit-gradationdata and outputs the resulting 8-bit-gradation data.

[0100] The reproducing process of the digital still camera 11 completesupon execution of step S16.

[0101] 3. Advantages etc. of First Embodiment

[0102] In the first embodiment, according to the above-describedprocess, 12-bit-gradation data is not recorded as it is but recorded as8-bit-gradation data and difference data. Therefore, the amount ofrecording data can be made smaller properly and easily than in a casewhere 12-bit-gradation data is compressed and recorded separately.

[0103] In the first embodiment, two gradation conversion characteristics(γ1 and γ2 or γ1 and γ3 in FIG. 4) coincide with each other in themiddle portion of the input signal range. Therefore, 8-bit-gradationdata and 12-bit-gradation data correspond to each other approximatelyone to one in the main input signal range. The compression efficiency ofdifference data can be increased more by making the correlation between8-bit-gradation data and 12-bit-gradation data stronger in this manner.

[0104] In the first embodiment, in image regions (high-luminance regionsor low-luminance regions) for which the visual sensitivity is low,difference data is recorded after being decimated. This makes itpossible to efficiently compress difference data in such a range thatthe decimation is not visually discernible.

[0105] In the first embodiment, difference data can be compressedefficiently by performing predictive coding, run-length coding, andentropy coding on it.

[0106] In the first embodiment, since the second gradation conversioncharacteristic is changed in such a manner as to be adapted to a featureof image data, 12-bit-gradation data that is more affluent in gradationcan be generated.

[0107] In the first embodiment, 12-bit-gradation data that is affluentin gradation can be reproduced when necessary based on recorded8-bit-gradation data and difference data.

[0108] Further, in the first embodiment, 12-bit-gradation data that hasbeen reproduced in the above manner is gradation-converted in asimplified manner and resulting gradation data is output. Therefore, auser can easily enjoy use of 12-bit-gradation data that is affluent ingradation with an external apparatus.

[0109] Next, another embodiment will be described.

[0110] Embodiment 2

[0111] A second embodiment is directed to a digital still camera.

[0112] The configuration of the digital still camera according to thesecond embodiment is the same as that of the digital still cameraaccording to the first embodiment (see FIG. 1). Therefore, the referencenumerals of the respective components shown in FIG. 1 will also be usedin the following description as they are and the configuration of thedigital still camera will not be described in the second embodiment.

[0113] 1. Image Recording Process

[0114]FIG. 5 is a flowchart showing an image recording process accordingto the second embodiment.

[0115] The image recording process according to the second embodimentwill be described below in order of step numbers shown in FIG. 5.

[0116] Step S21:

[0117] The A/D conversion unit 15 converts an image signal into14-bit-gradation data by linearly quantizing the image signal in adynamic range of 0%-400% white.

[0118] Step S22:

[0119] The CPU 18 converts the 14-bit-gradation data into8-bit-gradation data by performing gradation conversion of γ4 (see FIG.7) on it. In this case, it is preferable that the gradation conversionassure gradation reproduction up to about 140% white.

[0120] Step S23:

[0121] The CPU 18 converts the 14-bit-gradation data into12-bit-gradation data by performing gradation conversion of γ5 (see FIG.7) on it. In this case, it is preferable that the gradation conversionassure gradation reproduction up to 200% white (if possible, about 400%white).

[0122] Step S24:

[0123] The CPU 18 JPEG-compresses the 8-bit-gradation data and storescompressed data in an “image storage segment to be preferentiallyreferred to”, which is in an image file.

[0124] Step S25:

[0125] The CPU 18 JPEG-expands the compressed 8-bit-gradation data andobtains 8-bit-gradation expanded data (hereinafter referred to as “8-bitexpanded data”).

[0126] Step S26:

[0127] The CPU 18 sequentially refers to the 8-bit expanded data on apixel-by-pixel basis and judges whether the pixel being referred to islocated at a non-coincidence position. The term “non-coincidenceposition” means a position where the 8-bit part of the 12-bit-gradationdata does not coincide with the 8-bit expanded data. At such anon-coincidence position, difference data is necessary for reproductionof a 12-bit gradation. Conversely, at a coincidence position, differencedata is not necessary because 12-bit-gradation data can be reproducedfrom the 8-bit expanded data.

[0128] Specifically, if the 8-bit expanded data of the pixel beingreferred to exceeds a predetermined threshold value (indicated bycharacter A in FIG. 7), the CPU 18 judges that the pixel is located at anon-coincidence position because difference data is necessary forreproduction of 12-bit-gradation data. On the other hand, if the 8-bitexpanded data of the pixel being referred to is smaller than or equal tothe threshold value A, the CPU 18 judges that reproducing12-bit-gradation data from the 8-bit expanded data is sufficientpractically and hence judges that the pixel being referred to is notlocated at a non-coincidence position.

[0129] If the pixel being referred to is located at a non-coincidenceposition, the CPU 18 advances the process to step S27. On the otherhand, if the pixel being referred to is not located at a non-coincidenceposition, the CPU 18 advances the process to step S30.

[0130] Step S27:

[0131] For the pixel being referred to, the CPU 18 calculates adifference value between the 12-bit-gradation data and the 8-bitexpanded data.

[0132] Step S28:

[0133] The CPU 18 nonlinearly quantizes the difference value accordingto a characteristic for compressing high-luminance-side (orlow-luminance-side) gradations and thereby decreases the number ofquantization bits.

[0134] Step S29:

[0135] The CPU 18 further DPCM-compresses the difference value whosenumber of quantization bits has been decreased.

[0136] Step S30:

[0137] The CPU 18 judges whether all pixels have been referred to. Ifall pixel have been referred to, the CPU 18 advances the process to stepS31. On the other hand, if not all pixels have been referred to yet, theCPU 18 returns the process to step S26 to execute steps S26-S30 again.

[0138] Step S31:

[0139] The CPU 18 stores the compressed data of the difference data(i.e., an array of difference values that have been calculated in theabove manner) in an application segment in the image file.

[0140] Step S32:

[0141] The CPU 18 transfers the image file thus generated to the memorycard 20 and stores the image file therein.

[0142] The recording process according to the second embodimentcompletes upon execution of step S32.

[0143] 2. Image Reproducing Process

[0144]FIG. 6 is a flowchart showing an image reproducing processaccording to the second embodiment.

[0145] The image reproducing process according to the second embodimentwill be described below in order of step numbers shown in FIG. 6.

[0146] Step S41:

[0147] The CPU 18 extracts JPEG-compressed data from the image storagesegment of an image file stored in the memory card 20 and expands theJPEG-compressed data into 8-bit expanded data.

[0148] Step S42:

[0149] The CPU 18 sequentially refers to the 8-bit expanded data on apixel-by-pixel basis, and judges whether the pixel being referred to islocated at a non-coincidence position.

[0150] Specifically, if the 8-bit expanded data of the pixel beingreferred to is smaller than or equal to the threshold value A, the CPU18 judges that the pixel being referred to is not located at anon-coincidence position and advances the process to step S43. On theother hand, if the 8-bit expanded data of the pixel being referred toexceeds the threshold value A, the CPU 18 judges that the pixel beingreferred to is located at a non-coincidence position and advances theprocess to step S44.

[0151] Step S43:

[0152] The CPU 18 calculates 12-bit-gradation data from the 8-bitexpanded data of the pixel being referred to (with the characteristicshown in FIG. 7, the 8-bit expanded data may be employed as it is as12-bit-gradation data). Then, the CPU 18 advances the process to stepS47.

[0153] Step S44:

[0154] The CPU 18 extracts a difference value from difference data inthe application segment of the image file.

[0155] Step S45:

[0156] The CPU 18 restores an original difference value from theDPCM-compressed difference value. Further, the CPU 18 expands ahigh-luminance-side (or low-luminance-side) gradation of the differencevalue and thereby restores the corrected value by inverse-quantizationof the difference value.

[0157] Step S46:

[0158] The CPU 18 adds the restored difference value to the 8-bitexpanded data and thereby reproduces 12-bit-gradation data of the pixelbeing referred to.

[0159] Step S47:

[0160] The CPU 18 judges whether all pixels have been referred to. Ifall pixels have been referred to, the CPU 18 advances the process to thenext step S48. On the other hand, if not all pixels have been referredto, the CPU 18 returns the process to step S42 to execute steps S42-S47again.

[0161] Step S48:

[0162] The CPU 18 converts the 12-bit-gradation data thus calculatedinto 8-bit-gradation data by performing simplified gradation conversion(e.g., by sequentially performing conversion having a characteristicthat is reverse to a characteristic γ5 shown in FIG. 8 and conversionhaving a characteristic γ4′ shown in FIG. 8) for monitor display orprinting in an external apparatus on the 12-bit-gradation data, andoutputs the resulting 8-bit-gradation data.

[0163] The reproducing process of the digital still camera 11 completesupon execution of step S48.

[0164] 3. Advantages etc. of Second Embodiment

[0165] Also in the second embodiment, according to the above-describedprocess, 12-bit-gradation data is not recorded as it is but recorded as8-bit-gradation data and difference data. Therefore, the amount ofrecording data can be made smaller properly and easily than in a casewhere 12-bit-gradation data is compressed and recorded separately.

[0166] In the second embodiment, two gradation conversioncharacteristics (γ4 and γ5 in FIG. 7) coincide with each other up to thegradation value A. Therefore, the correlation between 8-bit-gradationdata and 12-bit-gradation data becomes stronger and hence thecompression efficiency of difference data can be increased more.

[0167] In the second embodiment, in a input signal range where thevisual sensitivity is low, difference data is recorded after beinglevel-compressed(nonlinearly quantized). This makes it possible torationally decrease the number of quantization bits of the differencedata in such a range that the compression is not visually discernible.

[0168] In the second embodiment, 8-bit-gradation data is recorded in an“image storage segment to be preferentially referred to”, which is in animage file. And difference data is recorded in the application segmentof the image file. This recording method makes it possible to reliablymaintain compatibility with conventional image files. Therefore,8-bit-gradation data in an image file according to the invention can beprocessed, displayed, or printed by using a general-purpose imagebrowsing program or a general-purpose external apparatus as it is.

[0169] In the second embodiment, at the time of image recording,difference data is generated based on 8-bit expanded data and12-bit-gradation data. Therefore, at the time of image reproduction,12-bit-gradation data can be reproduced more correctly based on the8-bit expanded data and the difference data.

[0170] In the second embodiment, at the time of reproduction, tertiarydata and primary data are correlated with each other by performing thenon-coincidence position judgment. With this operation, the tertiarydata need not include map data and an advantage is provided that thedata amount of the tertiary data can be reduced efficiently.

[0171] Tertiary data may be recorded so as to be divided into map dataof non-correlation regions and value data. In this case, compressed dataof a binary bit map indicating whether each pixel belongs to anon-correlation region may be recorded as the map data. Alternatively,data (e.g., chain coding data) indicating outline shapes ofnon-correlation regions or like data may be recorded as the map data.

[0172] Further, in the second embodiment, the data amount of differencedata may further be reduced by increasing the difference data samplingincrement. In addition, the reproduction performance of gradations ofsecondary data may further be increased by automatically changing thegradation conversion characteristic γ5 shown in FIG. 7 in accordancewith a feature of image data.

[0173] Supplements to Embodiments

[0174] For convenience of description, the above embodiments areexamples in which the invention is applied to a digital still camera.However, the invention is not limited to such a case and can be used inrecording or reproducing image data having a wide dynamic range. Forexample, an image processing program for execution of any of the aboveprocesses (e.g., the processes described above with reference to FIGS.2, 3, 5, and 6) may be written in a predetermined programming languageand recorded on a machine-readable recording medium.

[0175] The manner of practicing part of the invention relating to suchan image processing program is not limited to the above. For example, bydelivering data of such an image processing program via a communicationline, a “recording medium such as a memory or a hard disk on which theimage processing program is recorded” can be produced in a computer at adelivering destination. Further, it is possible to mediate a transfer ofsuch an image processing program or manufacture of a recording medium byinforming a party concerned about the location of the program.

[0176] The same operations and advantages as obtained by the aboveembodiments can be obtained on a computer by using such a recordingmedium.

[0177] It is possible to practice an image recording method and an imagereproducing method according to the procedures of the above processes(e.g., the processes described above with reference to FIGS. 2, 3, 5,and 6). Also in this case, the same operations and advantages asobtained by the above embodiments can be obtained.

[0178] Although the above embodiments are mainly directed to gradationprocessing, the invention is not limited to such a case. For example, inthe case of handling color image data, the above-described processingmay be performed for the gradation of each of stimulus values of RGB,YCbCr, etc.

[0179] For example, tertiary data may be calculated for only a componenthaving high luminous efficiency (e.g., Y or G) among stimulus values ofcolor image data. This makes it possible to further reduce the dataamount of tertiary data. In this case, at the time of reproduction,secondary data may be reproduced by using primary data and tertiary datathat is formed by a component having high luminous efficiency. Thisreproducing process cannot completely (100%) restore components (CbCr,RB, or the like to which the visual sensitivity is low) that are notincluded in the tertiary data. However, it is possible to reproducesecondary data in such a range that it is suitable for practical use interms of visual recognition.

[0180] Although the above embodiments use difference data as tertiarydata, the invention is not limited to such a case. In general, any datathat determines correlation between primary data and secondary data canbe used as tertiary data. For example, a ratio between primary data andsecondary data may be calculated as dissimilarity between them and usedas tertiary data.

[0181] In the first embodiment, tertiary data is compressed in as high adegree as possible by performing multiple coding, that is, performingpredictive coding, run-length coding, and entropy coding. However, theinvention is not limited to such a case. For example, tertiary data maybe compressed by performing one or two of predictive coding, run-lengthcoding, and entropy coding. Naturally, tertiary data may be recordedwithout being compressed.

[0182] In the above embodiments, 8-bit-gradation data is employed asprimary data and 12-bit-gradation data is employed as secondary data. Itgoes without saying that the numbers of gradations are unlimited tothose of 8 bits and 12 bits.

[0183] Although the above embodiments are directed to the case ofhandling a still image, the invention is not limited to such a case. Forexample, the invention may be applied to a case of handling a movingimage. In this case, for example, tertiary data may be compressed in thetime-axis direction.

[0184] Although in the above embodiments a generated image file isstored in a memory card, the recording unit of the invention is notlimited to being a unit for storing a file for a long period. Forexample, the recording unit of the invention may be a unit for storing afile temporarily in a system memory, an image memory, a buffer, or thelike.

[0185] The invention is not limited to the above embodiments and variousmodifications may be made without departing from the spirit and scope ofthe invention. Any improvement may be made in part or all of thecomponents.

What is claimed is:
 1. An image recording apparatus comprising: a firstconverting unit for converting image data into primary data having anN-bit range according to a first gradation conversion characteristic; asecond converting unit for converting the image data into secondary datahaving an M-bit range according to a second gradation conversioncharacteristic that is lower in the degree of level compression than thefirst gradation conversion characteristic or that causes no levelcompression, where M is greater than N; a correlation calculating unitfor calculating data that determines correlation between the primarydata and the secondary data and employing the calculated data astertiary data; and a recording unit for recording the primary data andthe tertiary data in a file.
 2. The image recording apparatus accordingto claim 1 , wherein the recording unit records the primary data in animage storage segment to be preferentially referred to, which is in thefile.
 3. The image recording apparatus according to claim 2 , whereinthe recording unit records the tertiary data in an application segmentoptionally able to be added, which is in the file.
 4. The imagerecording apparatus according to claim 1 , wherein the first gradationconversion characteristic and the second gradation conversioncharacteristic have the same characteristic curve in at least a part ofan entire input signal range.
 5. The image recording apparatus accordingto claim 1 , wherein the correlation calculating unit calculates datarelating to dissimilarity between the primary data and the secondarydata and employs the calculated data as the tertiary data.
 6. The imagerecording apparatus according to claim 1 , wherein the recording unitcompresses the tertiary data by nonlinearly quantizing it and recordsthe compressed tertiary data in the file.
 7. The image recordingapparatus according to claim 1 , wherein the recording unit compressesthe tertiary data by increasing a sampling increment of the tertiarydata on an image space and records the compressed tertiary data in thefile.
 8. The image recording apparatus according to claim 1 , whereinthe recording unit discriminates non-correlation regions that are imageareas where a substantial dissimilarity exists between the primary dataand the secondary data, and records the tertiary data in the file in amanner that the tertiary data is divided into map data indicating shapesof the non-correlation regions and data indicating values of thenon-correlation regions.
 9. The image recording apparatus according toclaim 1 , wherein the recording unit discriminates a non-coincidenceposition that is a position in an image where the secondary data cannotbe calculated directly from the primary data, and records the tertiarydata at the non-coincidence position in the file.
 10. The imagerecording apparatus according to claim 1 , wherein the recording unitcompresses the tertiary data by run-length coding, entropy coding,and/or predictive coding, and records the compressed tertiary data inthe file.
 11. The image recording apparatus according to claim 1 ,wherein the second converting unit changes the second gradationconversion characteristic in accordance with a feature of the imagedata.
 12. An image recording apparatus comprising: a first convertingunit for converting image data into primary data having an N-bit rangeaccording to a first gradation conversion characteristic; a secondconverting unit for converting the image data into secondary data havingan M-bit range according to a second gradation conversion characteristicthat is lower in the degree of level compression than the firstgradation conversion characteristic or that causes no level compression,where M is greater than N; a correlation calculating unit forcalculating data that determines correlation between the primary dataand the secondary data and employing the calculated data as tertiarydata; and a recording unit for recording the primary data and thetertiary data in a file, wherein said recording unit is a unit forirreversibly compressing the primary data and recording the irreversiblycompressed primary data, and said correlation calculating unit is a unitfor expanding the irreversibly compressed primary data, calculating datathat determines correlation between the expanded primary data and thesecondary data, and employing the calculated data as the tertiary data.13. The image recording apparatus according to claim 12 , wherein therecording unit records the primary data in an image storage segment tobe preferentially referred to, which is in the file.
 14. The imagerecording apparatus according to claim 13 , wherein the recording unitrecords the tertiary data in an application segment optionally able tobe added, which is in the file.
 15. The image recording apparatusaccording to claim 12 , wherein the first gradation conversioncharacteristic and the second gradation conversion characteristic havethe same characteristic curve in at least a part of an entire inputsignal range.
 16. The image recording apparatus according to claim 12 ,wherein the correlation calculating unit calculates data relating todissimilarity between the primary data and the secondary data andemploys the calculated data as the tertiary data.
 17. The imagerecording apparatus according to claim 12 , wherein the recording unitcompresses the tertiary data by nonlinearly quantizing it and recordsthe compressed tertiary data in the file.
 18. The image recordingapparatus according to claim 12 , wherein the recording unit compressesthe tertiary data by increasing a sampling increment of the tertiarydata on an image space and records the compressed tertiary data in thefile.
 19. The image recording apparatus according to claim 12 , whereinthe recording unit discriminates non-correlation regions that are imageareas where a substantial dissimilarity exists between the primary dataand the secondary data, and records the tertiary data in the file in amanner that the tertiary data is divided into map data indicating shapesof the non-correlation regions and data indicating values of thenon-correlation regions.
 20. The image recording apparatus according toclaim 12 , wherein the recording unit discriminates a non-coincidenceposition that is a position in an image where the secondary data cannotbe calculated directly from the primary data, and records the tertiarydata at the non-coincidence position in the file.
 21. The imagerecording apparatus according to claim 12 , wherein the recording unitcompresses the tertiary data by run-length coding, entropy coding,and/or predictive coding, and records the compressed tertiary data inthe file.
 22. The image recording apparatus according to claim 12 ,wherein the second converting unit changes the second gradationconversion characteristic in accordance with a feature of the imagedata.
 23. A recording medium on which an image processing program isrecorded, the image processing program comprising the steps of:converting image data into primary data having an N-bit range accordingto a first gradation conversion characteristic; converting the imagedata into secondary data having an M-bit range according to a secondgradation conversion characteristic that is lower in the degree of levelcompression than the first gradation conversion characteristic or thatcauses no level compression, where M is greater than N; calculating datathat determines correlation between the primary data and the secondarydata and employing the calculated data as tertiary data; and recordingthe primary data and the tertiary data in a file.
 24. A recording mediumaccording to claim 23 , wherein the image processing program furthercomprises the steps of: recording the primary data by irreversiblyconverting it; and expanding the irreversibly compressed primary data,and calculating data that determines correlation between the expandedprimary data and the secondary data, and employing the calculated dataas the tertiary data.
 25. An image recording method comprising the stepsof: converting image data into primary data having an N-bit rangeaccording to a first gradation conversion characteristic; converting theimage data into secondary data having an M-bit range according to asecond gradation conversion characteristic that is lower in the degreeof level compression than the first gradation conversion characteristicor that causes no level compression, where M is greater than N;calculating data that determines correlation between the primary dataand the secondary data and employing the calculated data as tertiarydata; and recording the primary data and the tertiary data in a file.26. The image recording method according to claim 25 , furthercomprising the steps of: recording the primary data by irreversiblyconverting it; and expanding the irreversibly compressed primary data,and calculating data that determines correlation between the expandedprimary data and the secondary data, and employing the calculated dataas the tertiary data.
 27. An image reproducing apparatus for reproducinga file generated by an image recording apparatus, the image recordingapparatus having: a first converting unit for converting image data intoprimary data having an N-bit range according to a first gradationconversion characteristic; a second converting unit for converting theimage data into secondary data having an M-bit range according to asecond gradation conversion characteristic that is lower in the degreeof level compression than the first gradation conversion characteristicor that causes no level compression, where M is greater than N; acorrelation calculating unit for calculating data that determinescorrelation between the primary data and the secondary data andemploying the calculated data as tertiary data; and a recording unit forrecording the primary data and the tertiary data in the file,comprising: a reading unit for reading the primary data and the tertiarydata from the file; and a secondary data calculating unit forreproducing the secondary data based on the primary data and thetertiary data.
 28. The image reproducing apparatus according to claim 27, wherein the secondary data calculating unit level-compresses thesecondary data so that the data has a bit range gradation-reproducibleby an external apparatus, and outputs the level-compressed data.
 29. Animage reproducing apparatus for reproducing a file generated by an imagerecording apparatus, having: a first converting unit for convertingimage data into primary data having an N-bit range according to a firstgradation conversion characteristic; a second converting unit forconverting the image data into secondary data having an M-bit rangeaccording to a second gradation conversion characteristic that is lowerin the degree of level compression than the first gradation conversioncharacteristic or that causes no level compression, where M is greaterthan N; a correlation calculating unit for calculating data thatdetermines correlation between the primary data and the secondary dataand employing the calculated data as tertiary data; and a recording unitfor discriminating a non-coincidence position that is a position in animage where the secondary data cannot be calculated directly from theprimary data and for recording the primary data and the tertiary data atthe non-coincidence position in the file, comprising: a reading unit forreading the primary data and the tertiary data from the file; and asecondary data calculating unit for reproducing the secondary data basedon the primary data and the tertiary data, wherein said secondary datacalculating unit discriminates the non-coincidence positions accordingto pixel values of the primary data, disposes the tertiary data at thenon-coincidence positions and performing positioning between the primarydata and the tertiary data, and reproduces the secondary data based onthe primary data and the tertiary data that corresponds to the primarydata in pixel position.
 30. A recording medium on which an imageprocessing program is recorded, for reproducing a file generated by animage recording apparatus, the image recording apparatus having: a firstconverting unit for converting image data into primary data having anN-bit range according to a first gradation conversion characteristic; asecond converting unit for converting the image data into secondary datahaving an M-bit range according to a second gradation conversioncharacteristic that is lower in the degree of level compression than thefirst gradation conversion characteristic or that causes no levelcompression, where M is greater than N; a correlation calculating unitfor calculating data that determines correlation between the primarydata and the secondary data and employing the calculated data astertiary data; and a recording unit for recording the primary data andthe tertiary data in the file, comprising the steps of: reading theprimary data and the tertiary data from the file; and reproducing thesecondary data based on the primary data and the tertiary data.
 31. Arecording medium on which an image processing program is recorded, forreproducing a file generated by an image recording apparatus, having: afirst converting unit for converting image data into primary data havingan N-bit range according to a first gradation conversion characteristic;a second converting unit for converting the image data into secondarydata having an M-bit range according to a second gradation conversioncharacteristic that is lower in the degree of level compression than thefirst gradation conversion characteristic or that causes no levelcompression, where M is greater than N; a correlation calculating unitfor calculating data that determines correlation between the primarydata and the secondary data and employing the calculated data astertiary data; and a recording unit for discriminating a non-coincidenceposition that is a position in an image where the secondary data cannotbe calculated directly from the primary data and for recording theprimary data and the tertiary data at the non-coincidence position inthe file, comprising the steps of: reading the primary data and thetertiary data from the file; and reproducing the secondary data based onthe primary data and the tertiary data, wherein said reproducingdiscriminates the non-coincidence positions according to pixel values ofthe primary data, disposes the tertiary data at the non-coincidencepositions and performing positioning between the primary data and thetertiary data, and reproduces the secondary data based on the primarydata and the tertiary data that corresponds to the primary data in pixelposition.
 32. An image reproducing method for reproducing a filegenerated by an image recording apparatus, having: a first convertingunit for converting image data into primary data having an N-bit rangeaccording to a first gradation conversion characteristic; a secondconverting unit for converting the image data into secondary data havingan M-bit range according to a second gradation conversion characteristicthat is lower in the degree of level compression than the firstgradation conversion characteristic or that causes no level compression,where M is greater than N; a correlation calculating unit forcalculating data that determines correlation between the primary dataand the secondary data and employing the calculated data as tertiarydata; and a recording unit for recording the primary data and thetertiary data in the file, comprising the steps of: reading the primarydata and the tertiary data from the file; and reproducing the secondarydata based on the primary data and the tertiary data.
 33. An imagereproducing method for reproducing a file generated by an imagerecording apparatus, having: a first converting unit for convertingimage data into primary data having an N-bit range according to a firstgradation conversion characteristic; a second converting unit forconverting the image data into secondary data having an M-bit rangeaccording to a second gradation conversion characteristic that is lowerin the degree of level compression than the first gradation conversioncharacteristic or that causes no level compression, where M is greaterthan N; a correlation calculating unit for calculating data thatdetermines correlation between the primary data and the secondary dataand employing the calculated data as tertiary data; and a recording unitfor discriminating a non-coincidence position that is a position in animage where the secondary data cannot be calculated directly from theprimary data and for recording the primary data and the tertiary data atthe non-coincidence position in the file, comprising the steps of:reading the primary data and the tertiary data from the file; andreproducing the secondary data based on the primary data and thetertiary data, wherein said reproducing discriminates thenon-coincidence positions according to pixel values of the primary data,disposes the tertiary data at the non-coincidence positions andperforming positioning between the primary data and the tertiary data,and reproduces the secondary data based on the primary data and thetertiary data that corresponds to the primary data in pixel position.